Interpretable Machine Learning

June 7, 2018

A client wants you to predict data scientist salaries with machine learning.

Let's predict data scientist salaries

What is Machine Learning

Machine learning is a method for teaching computers to make and improve predictions or behaviours based on data.

Step 1: Find some data

Kaggle conducted an industry-wide survey of data scientists. https://www.kaggle.com/kaggle/kaggle-survey-2017

Information asked:

Compensation
Demographics
Job title
Experience
…

Contains information from Kaggle ML and Data Science Survey, 2017, which is made available here under the Open Database License (ODbL).

Step 2: Throw ML on your data

Step 3: Profit.

Client: "There is a problem with the model!"

"What problem?""

Client: "Model predicts high salaries for old yet unskilled people."

Partial Dependence Plots

Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2013). Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation, 1–22. https://doi.org/10.1080/10618600.2014.907095

Friedman, J. H. (1999). Greedy Function Approximation : A Gradient Boosting Machine. North, 1(3), 1–10. https://doi.org/10.2307/2699986

Client: "We want to understand the model better!"

Permutation feature importance

Breiman, Leo. "Random forests." Machine learning 45.1 (2001): 5-32.

Gender?!

LIME

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Retrieved from http://arxiv.org/abs/1602.04938

Shapley

Some Theory

The Problem

Opaque decision making by machine learning algorithms
Bias and mistakes in the data
Trust as a user: Should I start a therapy with sever side effects, because the machine said so?
Debugging as a practitioner: Why did the algorithm miss-classify sample X? Did it learn generalizable features?

What is interpretability?

Interpretability is the degree to which a human can understand the cause of a decision. But we don't really have a good way to measure that really. Not as easy as benchmarking ML algorithms.

Interpretability is also a mean to look at the data and possible issues with them.

Miller, Tim. 2017. “Explanation in Artificial Intelligence: Insights from the Social Sciences.” arXiv Preprint arXiv:1706.07269.

When do we need interpretability?

We need interpretability when the loss function does not cover all constraints.

Decisions about humans.
Critical applications that decide about life and death.
Newly developed systems with unknown consequences.
Models using proxies instead of causal inputs.
Debugging the models.
Increasing trust.

Examples where we need interpretability

Credit scores
Autonomous vehicles
Medical recommendations
Automated diagnoses

When don't we need interpretability?

When we can capture everything in the loss function and the data collection. only causal relationships. perfect operationalization of features.

Things that work well. Well defined problems

Example: Digit recognition

What tools do we have?

Intrinsically interpretable models
Tools for analyzing specific black box models
Model-agnostic tools: analysing any black box model
Interpretability through making data explainable

Interpretable Models

Decision trees
Decision rules
Generalized regression models
…

Model-specific methods

RandomForestExplainer
Visualizing activations of neural networks

TODO: add images from papers

Model-agnostic methods

Partial dependence plots
LIME
…

Example-focused methods

Prototypes: What is the prototype of a certain class
Criticisms: Which data points are unusual for a certain class.
Counterfactuals: What do I need to change for an instance to change the prediction?
Anchors: What features do I have to anchor to keep prediction the same?

TODO: Cite

Outcomes of interpretability method:

Model
Data point
Interpretable unit (like a weight)
Summary statistic for a feature

Interested in learning more?

Read my book about "Interpretable Machine Learning" https://christophm.github.io/interpretable-ml-book/